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IMAGE PROCESSING DEPENDENT ON PRESENCE OR ABSENCE OF A 
PERSON AS THE IMAGE SUBJECT 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image processing method which is 
used to process a color image, an image processing apparatus which adopts the 
image processing method, and a storage medium which stores a program to 
achieve the image processing method. 
1 0 Related Background Art 

Usually, image data photographed or taken by a digital camera is stored 
as JPEG (Joint Photographic Expert Group) image data. Fig. 2 shows the flow 
of editing and printing for the JPEG image data. 

In Fig. 2, in a step S20 1 , the JPEG image data is read into a memory (the 
1 5 outline thereof will be described later). Then, in a step S202, a decode process 
is performed to the read JPEG image data to obtain eight-bit/three-component 
YCbCr data. In a step S203, a 3 x 3 matrix is applied to the YCbCr data to 
obtain eight-bit/three-component sRGB data. Continuously, in a step S204, the 
R, G and B values of the sRGB data are each displayed and corrected. Finally, 
20 in a step S205, a print process is performed to the corrected sRGB data. 

With the development of image pickup devices of the digital camera and 
image processing technology, the YCbCr data in tlie step S202 sometimes has a 
color gamut wider than that of an sRGB color space. In this case, a method of 
mapping any data that does not lie within the region of the sRGB color space into 
25 a boundary of the sRGB color space is often adopted to cope with this problem. 

On one hand, a method of using such a wide color gamut as it is in a print 
process is sometimes adopted. According to this method, in step S203, the 
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eight-bit YCbCr data is not converted into eight-bit sRGB data but converted into 
another eight-bit color space (hereinafter, called "xRGB data") having a color 
gamut wider than that of the sRGB color space, whereby the wide color gamut of 
the YCbCr data can be used in the print process as it is. However, in such a case, 
5 since the color gamut of the xRGB data is wider than that of the sRGB data 
though both the xRGB data and the sRGB data have eight bits, there occurs a 
problem that the tonality in the case of printing the xRGB data deteriorates as 
compared with a case of printing the sRGB data. Therefore, in the case of 
printing the xRGB data, when a photographic object (i.e., an object to be 

1 0 photographed) is a person, there may occur a problem that the tonality of skin 
(flesh color) of the person deteriorates. 

On the other hand, in a case of printing the sRGB data, clearness in a 
scenery image and the like is lacking, and there occurs a problem that the color 
intended by a user cannot be reproduced in the print process. 

1 5 For these reasons, the user has to perform the print process in a 

complicated way, by appropriately selecting the color space suitable for the 
processed image. 

SUMMARY OF THE INVENTION 

2 0 The present invention has been made in consideration of the above 

problems, and an object thereof is to provide an image processing method in 
which an appropriate color space is adopted according to a photographic object, 
an image processing apparatus to which the image processing method is applied, 
and a storage medium which stores a program to achieve the image processing 

2 5 method. An image processing method according to one aspect of the invention 
comprises determining whether or not input image data represents an image of a 
person as a subject of the image, and selecting a color space conversion condition 
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from among plural color space conversion conditions, including first and second 
color space conversion conditions, in accordance with the determination result 
obtained in the determining step. The second color space, which corresponds to 
the second color space conversion condition, has a color gamut wider than a first 
5 color space corresponding to the first color space conversion condition, and in a 
case where it is determined that the input image data represents the image of the 
person as the subject of the image, the second color space conversion condition is 
selected. The method also comprises performing the color space conversion on 
the input image data, using the selected color space conversion condition. 
1 0 The above and other objects of the present invention will become apparent 

from the following detailed description in conjunction with the attached drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram showing the system structure of the present 
1 5 invention; 

Fig. 2 is a flow chart for explaining a printing method of a JPEG image; 
Fig. 3 is a diagram showing the structure of a digital camera; 
Fig. 4 is a diagram showing the structure of a JPEG (Exif) file; 
Fig. 5 is a flow chart for explaining a process of performing a print 
2 0 process for each color space in accordance with additional information; 

Fig. 6 is a flow chart of a process of assigning image data for each color 

space; 

Fig. 7 is a flow chart of a process of assigning the image data for each 
color space; 

2 5 Fig. 8 is a flow chart of a process of performing a print process for each 

color space in accordance with an image analysis result; 

Fig. 9 is a flow chart of a process of assigning the image data for each 
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color space; and 

Fig. 10 is a flow chart of a process of assigning the image data for each 
color space. 

5 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, the preferred embodiments of the present invention will be 
explained in detail with reference to the attached drawings. Here, the 
embodiment of switching a color space to be used in a print process according to 
photographing mode information described in a header of a JPEG (Joint 

1 0 Photographic Expert Group) file (Exif/DCF file) will be explained. 
(First Embodiment) 

Fig. 1 is a block diagram schematically showing the structure of the first 
embodiment of the present invention. An input apparatus 101, which inputs 
instructions from a user and data, includes a pointing system such as a keyboard, a 

1 5 mouse and the like. A display apparatus 102 displays a GUI (Graphical User 

Interface), and a CRT, an LCD or the like is usually used for the display apparatus. 
A storage apparatus 103 stores image data and programs, and a hard disk is 
usually used for the storage apparatus. A CPU 104 performs an entire process of 
each above-mentioned apparatus. A ROM 105 and a RAM 106 provide 

2 0 necessary programs and data and a work area for the CPU 104. It is assumed 
that control programs necessary in processes indicated in the following flow 
charts are stored in the storage apparatus 103 or the ROM 105. In a case where 
the programs are stored in the storage apparatus 103, the programs are once read 
into the RAM 106 and then performed. 

25 It should be noted that, although an actual system includes various 

structural components other than such components as above, the explanation of 
these components will be omitted because they are not the central features of the 
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present invention. 

Fig. 3 is a diagram showing the schematic structure of a digital camera 
which records a photographed image after performing JPEG compression. A 
digital (still) camera 301 is equipped with a mode dial 303, whereby a mode in 
5 photographing can be changed by rotating the mode dial 303. Moreover, there 
are four kinds of photographing modes which consists of a person photographing 
mode 307 for photographing a person, a scene photographing mode 308 for 
photographing the scene, a night scene photographing mode 309 for 
photographing the night scene, and an automatic photographing mode 306 used 
1 0 for photographing the most part of the scene. These modes can be changed 
every time the mode dial 303 is rotated, and the currently selected mode is 
displayed as an icon of a selection mode 305 on an LCD panel 304. When 
depressing a shutter button 302, a JPEG image is photographed in the currently 
selected mode. 

1 5 Fig. 4 is a diagram schematically showing the structure of the JPEG 

image photographed by the digital camera 301. The JPEG image photographed 
by the digital camera 301 is usually stored as a file format called an Exif. A data 
description space called APPl (OxFFEl) Marker exists in a header of an Exif file 
401, and information in photographing (model name, photographing time, 

2 0 photographing mode or the like) is described as Exif tag information 402 in APPl 
Marker. 

Fig. 6 is a flow chart showing a process of judging whether the JPEG 
image is to be emphasized in tonality or color gamut, referring to the 
photographing mode. In a step S601, a tonality emphasis bit flag of judging 
2 5 whether or not the JPEG image is to be emphasized in tonality is formed in the 
RAM 106, and that flag is initialized to "0" (OFF). In a step S602, the 
photographing mode is referred to by accessing to the JPEG image and the value 
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of the mode is read. In a step S603, when the read photographing mode 
corresponds to a person mode, the flow advances to a step S604, and when the 
read photographing mode corresponds to modes other than the person mode, the 
process ends. In step S604, the tonality emphasis bit flag is rewritten to "1" 
5 (ON) and then the process ends. Thus, when the photographing mode 

corresponds to the person mode, it is judged that the image is to be emphasized in 
tonality, and when the photographing mode corresponds to modes other than the 
person mode, it is judged that the image is to be emphasized in color gamut. 

Fig. 5 is the tlow chart showing a process of actually printing the JPEG 

1 0 image. In a step S501, the JPEG image data is read on the RAM 106 to decode 
it into eight-bit YCbCr data. The decoded YCbCr data is expanded in the RAM 
106. In a step S502, an Exif tag is analyzed from the JPEG image data read in 
the RAM 106. In a step S503, it is judged whether or not the JPEG image data 
is to be emphasized in tonality from the analyzed Exif tag. This judgment 

1 5 process is the same as that explained above with reference to Fig. 6. 

When the image is to be emphasized in tonality, the flow advances to a 
step S504, while when the image is to be emphasized in color gamut, the flow 
advances to a step S507. In step S504, the eight-bit YCbCr data is converted 
into eight-bit sRGB data. For this conversion, although a 3 x 3 matrix is 

2 0 generally used, another method of using a conversion table or an ICC profile is 
also available. In a step S505, the image data expanded into the sRGB data is 
corrected (modified) by using an automatic correction module or application. 
This correction may be automatically performed or manually performed by using 
the application software by a user. Here, it should be noted that any method is 

2 5 available in the correction. 

In a step S506, the eight-bit sRGB data is converted into eight-bit 
DeviceRGB data to perform a color matching for the print. For this conversion. 
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although a conversion table is generally used, another method of using the ICC 
profile or the like is available. In a step S510, the DeviceRGB data is converted 
into CMYK data matching with the print. In a step S5 1 1 , the print is performed 
on the basis of the data converted into the CMYK data. In step S507, since it is 
5 judged that the image is to be emphasized in color gamut, the eight-bit YCbCr 
data is converted into eight-bit xRGB data, this data being in a color space that has 
a color gamut than that of an sRGB color space. For this conversion, although a 
3x3 matrix is generally used, another method of using the conversion table, the 
ICC profile or the like is also available. In a step S508, the image data expanded 

1 0 into the xRGB data is corrected (modified) by using the automatic correction 
module or the application software. This correction may be performed 
automatically, or manually by using the application software by a user. It should 
be noted that any known method can be used for this correction. However, in a 
case of xRGB data, since there sometimes is a case where the image data cannot 

15 be displayed on the display apparatus 102, when the correction is manually 

performed by the user, the correction has to be performed under the condition that 
virtual color is displayed on the display apparatus 102. 

In a step S509, the eight-bit xRGB data is converted into eight-bit 
DeviceRGB data to perform a color matching for the print. For this conversion, 

2 0 although a conversion table is generally used, another method of using the ICC 
profile or the like is also available. 

The process is sequentially continued to advance to the steps S510 and 
S5 1 1 after the processes in the steps S506 and S509 have been performed. 

In this manner, it is automatically judged whether the image is to be 

2 5 emphasized in tonality or color gamut, by referring to the photographing mode of 
the Exif tag, thereby enabling one to perform the printing in a color space suitable 
for the particular image in question. It should be noted that the data called 



- 8 - Substitute specification (clean) 

AM 10/620,688 
Atty. Docket No. 03500.017429 



"xRGB" indicates a color space having an arbitrary space wider than that of the 
sRGB data, and any color space such as AdobeRGB(TM) or the like may be used 
as xRGB if it has a wider gamut than that of the sRGB data. 

In the present embodiment, the data is converted into eight-bit sRGB data 
5 only when the photographing mode corresponds to the person mode. However, 
it is available to convert the data into the sRGB data for another photographing 
mode emphasized in tonality. 

It is also possible to set the correspondence between the photographing 
mode and color space conversion in accordance with the instruction from the user. 
1 0 (Second Embodiment) 

Now, a second embodiment, where a JPEG image is printed upon judging 
whether the image is to be emphasized in tonality or color gamut from a 
photographing mode and other photographing information, will be explained. 
Since the system structure is the same as that of the first embodiment, explanation 
1 5 thereof will be omitted in the present embodiment. 

Fig. 7 is a flow chart showing a process of determining if the image is to 
be emphasized in tonality or the color gamut by referring to the photographing 
mode and flash information being one of Exif tag information (photographing 
information). In a step S701, a tonality emphasis bit flag, indicating whether or 
2 0 not the image is to be emphasized in tonality is formed in the RAM 106, and that 
flag is initialized to "0" (OFF). In a step S702, photographing mode information 
is referred by accessing to the JPEG image and a value of the mode information is 
read. 

In a step S703, when the photographing mode corresponds to a person 
2 5 mode, the flow advances to a step S704, while when the photographing mode 

corresponds to modes other than the person mode, the process ends. In the step 
S704, the flash information is referred to. In a step S705, in a case of the flash- 
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ON, it is judged that the image is to be emphasized in color gamut, and the 
process ends. This is because a high possibility of a white blank (or white dot) 
for flesh color of a person being a photographic object is anticipated in a print 
process according to an sRGB color space in case of the flash-ON. In this case, 
5 the printing for a region of the white blank can be appropriately performed by the 
print according to an xRGB color space. On the other hand, in a case of the 
flash-OFF, the tonality emphasis bit flag is rewritten to "1" (ON) in a step S706, 
and then the process ends. In this manner, by using the photographing mode and 
the flash infomiation, it is judged whether the image is to be emphasized in 

1 0 tonality or in color gamut. 

Since the process of performing the print is the same as that in the first 
embodiment, explanation thereof will be omitted in the present embodiment. 

In this manner, it is Judged whether the image is to be emphasized in 
tonality or color gamut by utilizing the photographing mode and other 

1 5 photographing information, and it becomes possible to perform the printing by 

means of a color space matching the particular image in question, according to the 
judgment result. 
(Third Embodiment) 

Now, there will be explained an embodiment in which a JPEG image is 

2 0 printed upon judging whether the image is to be emphasized in tonality or color 
gamut from additional information other than Exif photographing information. 
Since the system structure is the same as that of the first embodiment, explanation 
thereof will be omitted in the present embodiment. 

As the additional information to be added to the JPEG image, there are 

2 5 not only the Exif information described in the foregoing embodiments but also 

keywords of describing photographic object information, place information and/or 
event information, which are mainly utilized in the search. Among the above 
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information, a person name can be described in the photographic object 
information. That is, from an inverse viewpoint, if the person name is added as 
a keyword, it is obvious that the person is recorded in the image. 

Fig. 10 is the flow chart showing a process of determining if the image is 
5 to be emphasized in tonality or color gamut by referring to the keyword in the 
additional information. In a step SI 001, a tonality emphasis bit flag of judging 
whether or not the image is to be emphasized in tonality is formed on the RAM 
106, and that flag is initialized to "0" (OFF). In a step S1002, the keyword in the 
additional information is referred to by accessing to the JPEG image and a value 
10 of the information is read. 

In a step SI 003, in a case where the person name is included in the read 
keyword, the flow advances to a step S1004, and in other cases, the process ends. 
In step S1004, the tonality emphasis bitflag is rewritten to "1" (ON), and then the 
process ends. 

15 In this manner, when the person name is included in the keyword in the 

additional information, it is judged that the image is to be emphasized in tonality, 
and when no person name is included in the keyword in the additional information, 
it is judged that the image is to be emphasized in color gamut. 

Since the process of performing the printing is the same as that in the first 

2 0 embodiment, explanation thereof will be omitted in the present embodiment. 

In this manner, it can be judged whether the image is to be emphasized in 
tonality or color gamut from the additional information other than the 
photographing information, and it becomes possible to perform the printing by 
means of a color space suitable for the particular image in question. At this time, 

25 it is also possible to judge whether the image is to be emphasized in tonality or 
color gamut by combining the photographing information with the other 
additional information. For example, if the photographing mode is set to an 
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automatic mode, the image essentially has to be emphasized in color gamut. 
However, it is considered that in a case where the person name is included in the 
keyword in the additional information, the image is set to be emphasized in 
tonality. 
5 (Fourth Embodiment) 

There will now be described an embodiment in which a JPEG image is 
printed upon judging whether the image is to be emphasized in tonality or color 
gamut by an analysis of the JPEG image without using additional information of 
the JPEG image. Here, since the system structure is the same as that of the first 
1 0 embodiment, explanation thereof will be omitted in the present embodiment. 

Fig. 9 is a flow chart showing a process of performing face recognition 
for an image and determining if the image is to be emphasized in tonality or color 
gamut according to the recognition result. 

In a step S901, a tonality emphasis bit flag indicating whether or not the 
1 5 image is to be emphasized in tonality is formed in the RAM 106, and that flag is 
initialized to "0" (OFF). In a step S902, a face recognition process is performed 
on the image by accessing the JPEG image. 

When a face is recognized in a step S903, the flow advances to a step 
S904, and in other cases, the process ends. In step S904, the tonality emphasis 
2 0 bit flag is rewritten to " 1 " (ON), and then the process ends. 

In this manner, if the face is recognized as a result of the face recognition 
process, it is judged that the image is to be emphasized in tonality, and if no face 
is recognized, it is judged that the image is to be emphasized in color gamut. 

Fig. 8 is a flow chart showing a process of actually printing the JPEG 
2 5 image. In a step S801, the JPEG image data is read in the RAM 106, to decode 
it into eight-bit YCbCr data. The decoded YCbCr data is expanded in the RAM 
106. In a step S802, a face region is detected from the JPEG image data read on 
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the RAM 106. 

In a step S803, it is judged whether or not the image is to be emphasized 
in tonality, from a result of the face detection. The process of judgment is the 
same as the process mentioned abo\ e with reference to Fig. 9. In a case of the 
5 tonality emphasis, the flow advances to a step S804, and in a case of the color 
gamut emphasis, the flow advances to a step S807. Since the process to be 
performed thereafter is the same as that shown in Fig. 5 (that is, steps S805, S806 
and S808 - S81 1 are the same as steps S505, S506 and S508 - S51 1, respectively), 
explanation thereof will be omitted in the present embodiment. 
10 In this manner, it can be judged whether the image is to be emphasized in 

tonality or color gamut from information other than the additional information of 
the image, and it becomes possible to perform the printing using a color space that 
suits the particular image in question. 

As means for analyzing the image, it is possible to use not only means for 
1 5 recognizing the face but also any means of capable of specifying, by detecting 
flesh color, that the photographic object is a person. 

The embodiments of the present invention are explained by an example 
of the JPEG image. However, it is needless to say that another image format is 
available in an aspect of switching a color space to be used in printing according 
20 to the additional information. 

As described above, the present invention has been explained on the basis 
of the preferable embodiments. However, the present invention is not limited to 
these embodiments, but may be modified in various maimers within the scope of 
the claims. 



25 



